Perfect information stochastic games and related classes

For n-person perfect inlbrmation stochastic games and for n-person stochastic games with Additive Rcwards and Additive Transitions (ARAT) we show the existence of pure limiting average equilibria. Using a similar approach we also derive the existence of limiting average e-equilibria for two-person switching control stochastic games. The orderfield property holds for each of the classes mentioned, and algorithms to compute equilibria are pointed out. 1 Model We deal with n-person stochastic games with finite state and action spaces. Only for some special classes of stochastic games limiting average e-equilibria are known to exist, but generally their cxistence remains to be an open problem (see Thuijsman [ 19921 for a survey on equilibrium existence). For zerosum stochastic games Mertens & Neyman [198 lJ showed the existence of the limiting average value. Approaching this value generally involves the use of history dependent strategies. In this paper we show existence of limiting average equilibria (in 'almost stationary' behavior strategies) for perfect information stochastic games and for stochastic games with Additive Rewards and Additive Transitions (ARAT). For two-person stochastic games with switching control we show the existence of limiting average e-equilibria. For none of these related classes (any perfect information game has ARAT as well as switching control structure) equilibrium existence was known before. Our method implies that the orderfield property holds for these classes, i.e. if payoffs and transitions are rational, then there are rational equilibrium strategies and rational equilibrium rewards as well. Algorithms to determine equilibria are also pointed out. Whenever we speak 1 We gratefully acknowledge valuable remarks by J. Flesch and by anonymous referees on earlier versions.